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□  We  establish  a  heavy  traffic  limit  theorem  for  the  queue-length  process  in  a  critically  loaded  single 
class  queueing  network  with  state-dependent  arrival  and  service  rates.  A  distinguishing  feature 
of  our  model  is  non-Markovian  state  dependence.  The  limit  stochastic  process  is  a  continuous- 
path  reflected  process  on  the  nonnegative  orthant.  We  give  an  application  to  a  generalized  Jackson 
network  with  state-dependent  rates. 

Keywords  Diffusion  approximation;  Non-Markovian  networks;  State-dependent  net¬ 
works;  Weak  convergence. 

Mathematics  Subject  Classification  Primary  60F17;  Secondary  60K25,  60K30,  90B15. 


1.  INTRODUCTION 

Queueing  systems  with  arrival  and  (or)  service  rates  depending  on  the 
system’s  state  arise  in  various  application  areas,  which  include  manufac¬ 
turing,  storage,  service  engineering,  and  communication  and  computer 
networks.  When  human  servers  are  involved,  longer  queues  may  lead  to 
customers  being  discouraged  to  join  the  queue  or  may  affect  productivity. 
(We  provide  more  detail  in  Section  5.)  State-dependent  features  are  present 
in  congestion  control  protocols  in  communication  networks,  such  as  TCP 
(see  Refs,!1-.!. ii,i4,20]  arKj  references  therein). 

In  this  paper,  we  consider  an  open  network  of  single  server  queues  with 
the  arrival  and  service  rates  depending  on  the  queue  lengths.  The  network 
consists  of  K  single-server  stations  indexed  1  through  K.  Each  station  has  an 
infinite  capacity  buffer  and  the  customers  are  served  according  to  the  first- 
in-first-out  discipline.  The  arrivals  of  customers  at  the  stations  occur  both 
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externally,  from  the  outside,  and  internally,  from  the  other  stations.  Upon 
service  completion  at  a  station,  a  customer  is  either  routed  to  another  station 
or  exits  the  network.  Every  customer  entering  the  network  eventually  leaves 
it.  A  distinguishing  feature  of  the  model  is  non-Markovian  state  dependence. 
More  specifically,  the  number  of  customers  at  station  i,  where  i  =  1 ,  2, . . . ,  K, 
is  governed  by  the  following  equations: 

Qj(t)  =0,(0)  +Mt)  +Bi(t)  —  Di  ( t) , 

Ml)  =Nf(J\i«l(s))dsY 

K  (1 1) 
MO  =£> >ji{Dj(t)), 
j= 1 

Did)  =N?(J  ^(aW)l{Q,fi)>0}^). 

where  £3(.s)  =  (Qj(s),  . . . ,  Qfc(s))  denotes  the  vector  of  the  queue  lengths 
at  the  stations  at  time  5.  The  quantities  N^(t)  and  N^it)  represent  the 
number  of  exogenous  arrivals  and  the  maximal  number  of  customers  that 
can  be  served,  respectively,  at  station  i  by  time  t  under  “nominal”  conditions, 
<£ '  ji(m)  represents  the  number  of  customers  routed  from  station  j  to  station 
i  out  of  the  first  m  customers  served  at  station  j,  and  and  fii(Q(t)) 

represent  instantaneous  exogenous  arrival  and  service  rates,  respectively,  for 
station  i  at  time  /  given  the  queue  length  vector  Q(l) .  Thus,  At(l)  represents 
the  cumulative  number  of  exogenous  arrivals  by  time  t  at  station  i,  Dj{t) 
represents  the  cumulative  number  of  departures  by  time  t  from  station  i, 
and  Bj  ( l)  represents  the  cumulative  number  of  customers  routed  to  station 
i  from  the  other  stations  by  time  t.  The  quantities  <3,  (0),  N^(t),  N^{t),  and 
\j(m)  are  referred  to  as  network  primitives.  Generalized  Jackson  networks 
is  a  special  case  of  (1.1)  where  the  Nf1  and  are  renewal  processes,  the 
are  Bernoulli  processes,  and  A .,(•)  =  /x,  (-)  =  1. 

Our  goal  is  to  obtain  limit  theorems  in  critical  loading  for  the  queue 
length  processes  akin  to  diffusion  approximation  results  available  for  gener¬ 
alized  Jackson  networks;  see  Reiman^19].  Such  limit  theorems  are  useful  to 
obtain  approximations  to  various  quantities  of  interest.  For  instance,  using 
the  limit  stochastic  processes,  one  can  approximate,  via  either  numerical  or 
analytical  methods,  the  mean/variance  of  the  queue  length  or  the  fraction 
of  idle  time  for  particular  service  stations  (cf.  Refs.  1 1 11 2  J . 

The  results  on  the  heavy  traffic  asymptotics  for  state-dependent  rates 
available  in  the  literature  are  confined  mosdy  to  the  case  of  diffusion  lim¬ 
its  for  Markovian  models;  see  Yamada'21 Mandelbaum  and  Pats' 11 and 
Chapter  8  of  Kushner^11^.  Yamada^"1^  and  Mandelbaum  and  Pats^14^  draw 
on  the  work  of  Krichagina^10^,  who  studies  a  Markovian  closed  network  with 
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state-dependent  rates.  They  consider  a  case  of  the  model  (1.1),  where  the 
primitive  arrival  and  service  processes  are  standard  Poisson.  Kushner' 1 1  also 
includes  a  treatment  of  that  model  (see  Theorem  2.1,  p.  318);  however, 
their  basic  assumptions  are  formulated  in  terms  of  the  conditional  distri¬ 
butions  of  the  interarrival  (or  service)  intervals  (or  the  routing),  given  the 
“past.”  Those  authors  obtain  results  in  which  the  drift  coefficients  of  the 
limit  diffusion  processes  are  state-dependent  and  the  diffusion  coefficients 
may  be  either  constant  or  state-dependent,  which  is  determined  by  the  scal¬ 
ing  used.  Yarnada'21  and  Kus  liner' 1 1  assume  critical  loading  and  obtain 
mostly  diffusions  with  state-dependent  diffusion  coefficients,  although  Ya- 
mada^21'  considers  an  example  with  a  constant  diffusion  coefficient  where 
the  drift  has  to  be  linear  and  Theorem  2.1  on  p.  318  of  Kushner' 1 1 '  con¬ 
cerns  the  case  of  constant  diffusion  coefficients.  In  contrast,  Mandelbaum 
and  Pats ' 11  do  not  restrict  their  analysis  to  critical  loading,  and  their 
limits  have  constant  diffusion  coefficients.  Mandelbaum  and  Pats' 1 1 '  and 
Kushner' 1 1 '  also  allow  the  process  of  routing  the  customers  inside  the  net¬ 
work  to  be  state-dependent;  however,  their  reasoning  seems  to  be  unsub¬ 
stantiated,  as  discussed  in  Section  2.  Section  7  of  Yamada'21'  is  concerned 
with  a  non-Markovian  case  where  the  processes  are  standard  Poisson, 
the  processes  N. f  are  renewal  processes,  and  /x;(0  =  1.  It  is  also  mentioned 
that  an  extension  to  the  case  of  renewal  arrivals  with  A .,(•)  =  1  and  standard 
Poisson  processes  Nf  is  possible. 

The  main  contribution  of  this  piece  of  work  is  incorporating  general 
arrival  and  service  processes.  This  is  achieved  by  applying  an  approach  dif¬ 
ferent  from  the  one  used  by  Yamada^21^,  Mandelbaum  and  Pats'14",  and 
Kushner' 11 '.  The  proofs  of  those  authors  rely  heavily  on  the  martingale  weak 
convergence  theory.  They  are  quite  involved,  on  the  one  hand,  and  do  not 
seem  to  be  easily  extendable  to  more  general  arrival  and  service  processes, 
on  the  other  hand.  In  our  approach,  we,  in  a  certain  sense,  return  to  the 
basics  and  employ  ideas  that  have  proved  their  worth  in  the  setup  of  general- 
izedjackson  networks.  We  show  that  continuity  considerations  may  produce 
stronger  conclusions  at  less  complexity.  Our  main  result  states  that  if  the 
network  primitives  satisfy  certain  limit  theorems  with  continuous-path  lim¬ 
its,  then  the  multidimensional  queue-length  processes,  when  suitably  scaled 
and  normalized,  converge  to  a  reflected  continuous-path  process  on  the 
nonnegative  orthant.  If  the  limits  of  the  primitives  are  diffusion  processes, 
the  limit  stochastic  process  is  a  reflected  diffusion  with  state-dependent  drift 
coefficients  and  constant  diffusion  coefficients.  The  scaling  we  use  does  not 
capture  the  case  of  state-dependent  diffusion  coefficients.  We  also  give  an 
application  to  generalizedjackson  networks  with  state-dependent  rates,  thus 
providing  an  extension  of  Reiman’s'19"  results  and  particularise  further  by 
looking  at  a  tandem  queue  of  human  servers.  In  addition,  we  bridge  certain 
gaps  in  the  reasoning  of  Yamada'21",  Mandelbaum  and  Pats'14",  and  Kush¬ 
ner'11".  The  proofs  in  Refs.'21,14,11"  assume  that  the  functions  A .;(•)  and  Hi(') 
are  bounded  and  allude  to  a  “truncation  argument”  for  the  unbounded 
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case  omitting  the  details.  In  partictilar,  existence  and  uniqueness  for  (1.1) 
is  not  fully  addressed.  We  prove  both  the  existence  and  uniqueness  of  a 
solution  to  equations  (1.1)  and  the  limit  theorem  under  linear  growth  con¬ 
ditions.  A  more  detailed  discussion  of  earlier  results  is  provided  at  the  end  of 
Section  2. 

A  different  class  of  results  on  diffusion  approximation  concerns  queue¬ 
ing  systems  modeled  on  the  many-server  queue  with  a  large  number  of 
servers.  In  such  a  system  the  service  rate  decreases  to  zero  gradually  with 
the  number  in  the  system — whereas  in  the  model  considered  here,  it  has  a 
jump  at  zero  (see  (2. Id) ) — so  the  limit  process  is  an  unconstrained  diffusion; 
see  Mandelbaum,  Massey,  and  Reiman^13!,  Pang,  Talreja,  and  Wliitt^16^,  and 
references  therein.  We  do  not  consider  those  setups  in  this  paper. 

The  exposition  is  organised  as  follows.  In  the  next  section,  we  state  and 
discuss  our  main  result.  The  proof  is  provided  in  Section  3.  In  Section  4,  an 
application  to  state-dependent  generalized  Jackson  networks  is  presented. 
Section  5  spells  out  the  process  for  a  tandem  queue.  The  appendix  contains 
a  proof  of  the  pathwise  queue-length  construction  underlying  the  definition 
of  the  model. 

Some  notational  conventions  are  in  order.  All  vectors  are  understood  as 
column  vectors,  \x\  denotes  the  Euclidean  length  of  a  vector  x,  its  compo¬ 
nents  are  denoted  by  x,-,  unless  mentioned  otherwise,  superscript  T  is  used 
to  denote  the  transpose,  1a  stands  for  the  indicator  function  of  an  event 
A,  Sij  represents  Kronecker’s  delta,  \_a\  denotes  the  integer  part  of  a  real 
number  a,  IN  denotes  the  set  of  natural  numbers,  and  Z+  denotes  the  set 
of  nonnegative  integers.  We  use  O([0,  oo),  Me)  to  represent  the  Skorohod 
space  of  right  continuous  ]Re -valued  functions  with  left  hand  limits  which  is 
endowed  with  the  Skorohod  topology,  =4>  represents  convergence  in  distri¬ 
bution  of  random  elements  with  values  in  an  appropriate  metric  space;  see 
Billingsley ^  and  Ethier  and  Kurtz ' 6 '  for  more  information.  We  also  recall 
that  a  sequence  Vn  of  stochastic  processes  with  trajectories  in  a  Skorohod 
space  is  said  to  be  C-tight  if  the  sequence  of  the  laws  of  the  Vn  is  tight,  and  if 
all  limit  points  of  the  sequence  of  the  laws  of  the  Vn  are  laws  of  continuous- 
path  processes  (see,  e.g.,  Definition  3.25  and  Proposition  3.26  in  Chapter  VI 
ofjacod  and  Sfiiryaev191) . 


2.  THE  MAIN  RESULT 

Let  (f^,^7,  P)  be  a  probability  space  on  which  all  random  variables 
considered  in  this  paper  are  assumed  to  be  defined.  We  consider  a  se¬ 
quence  of  networks  indexed  by  n  with  a  similar  structure  to  the  one  de¬ 
scribed  in  the  Introduction.  For  the  w-th  network  and  for  i  e  IK,  where 
K  =  {1,  2,  . . . ,  K},  let  A”(t)  represent  the  cumulative  number  of  customers 
that  arrive  at  station  i  from  outside  the  network,  let  B"(t)  represent  the 
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number  of  endogeneous  arrivals  at  station  i,  and  let  D”(l)  represent 
the  cumulative  number  of  customers  that  are  served  at  station  i  during 
the  time  interval  [0,  t\.  Let  J  c  K  represent  the  set  of  stations  with  ac¬ 
tual  exogenous  arrivals  so  that  A'-(t)  =  0  if  i  &  J.  We  call  A"  =  (A",  i  e  K) 
and  Dn  =  (j D‘\  i  e  K),  where  A”  =  ( Af(t),t  >  0)  and  D”  =  t>  0), 

the  arrival  process  and  service  process  for  the  w-th  network,  respectively. 
We  associate  with  the  stations  of  the  network  the  processes  O”  =  (<f>"  ,  j  G 
K),  i  e  K,  where  <$>”■  —  (<f>L(m),  m  —  1,  2,  . . .),  and  <!>'',(  m)  denotes  the 
cumulative  number  of  customers  among  the  first  m  customers  departing 
station  i  that  go  directly  to  station  j.  The  process  T"  =  ( <b ■ ,  i,  j  e  K)  is 
referred  to  as  the  routing  process.  The  state  of  the  network  at  time  t  is  de¬ 
scribed  by  Qn(t)  =  . . . ,  Q£(t)),  where  QJ'(l)  represents  the  number 

of  customers  at  station  i  at  time  t. 

Precisely,  the  model  is  defined  as  follows.  Let  A."  and  //",  where  i  e  IK, 
be  Borel  functions  mapping  1R+  to  1?+,  with  A”(x)  =0  if  i  J ,  and  let 
A”  =  (A”,  i  e  K)  and  /z”  =  (/x”,  i  e  K).  These  functions  have  the  mean¬ 
ing  of  state-dependent  arrival  and  service  rates.  Let  Nf"n  =  ,  t  > 

0)  and  N f'"  —  (N^'n(t)  ,  t  >  0)  represent  nondecreasing  Z+-valued  pro¬ 
cesses  with  trajectories  in  the  Skorohod  space  D([0,  oo),  1R)  such  that 
N^'n(0)  —  N?’n(0)  =  0  and  the  jumps  of  N®’n  are  of  size  one.  We  define 
Nf'n(t)  =  [/J  if  i  $  J .  (The  latter  is  but  a  convenient  convention.  Since 
A"(x)  =  0  if  i  $  J ,  the  process  Nf’’n  is  immaterial,  as  the  equations  below 
show.)  Let  ((t>l‘-(m)  ,  m  e  iV)  be  Z+-valued  non  decreasing  processes  such 

that  ^j(m)  =  m  and  let  (A"(ti)  be  Z+-valued  random  variables.  The 

processes  A”,  IK,  Df,  and  QJ1  are  defined  as  Z+-valued  processes  with  trajec¬ 
tories  in  D([0,  oo),  1R)  satisfying  a.s.  the  equations 


Q?(t)  =  Qf( 0)  +A”(t)  -  D?(t), 

(2.1a) 

A”(t )  =  Nf-n  ^ 

f  K(Q.n(*))ds j, 

(2.1b) 

K 

i= i 

(2.1c) 

Df(t)  =  N?’n  ^ 

J  Ki(Q.n(s))^{Q’,(s)>0}d^j  > 

(2. Id) 

where  t  >  0  and  i  e  IK.  Accordingly,  Qn,  An,  and  D"  are  random  elements  of 
O([0,  oo),  Mk).  (Existence  and  uniqueness  for  (2.1a)-(2.1d)  are  addressed 
in  Lemma  2.1.) 
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Let  P  —  (pip  i,  j  e  A)  be  a  substochastic  matrix,  R  =  I  —  PT,  where 
/  denotes  the  K  x  /^-identity  matrix,  and  pi  =  (p ,  j  e  A").  We  denote 


Q”(0) 

Q"(0) 

Af”(0 

N'/'n{nl)  —  nt 

’ 

■s/n 

N®’n(nt)  —  nt 

^(0 

d>”(L^fJ)  -  pint 

■s/n 

■s/n 

ivf" 

=  (Nfn(t),  t>  0), 

NAn 

=(Nfn,  i  e  A), 

=  (#?’"(*),  t>  0), 

N°-n 

=(Nfn,  i  e  A), 

— n  n .  .  —  n  n  . 

<f>.  =(3y  (t),  t  >  0),  $  =(Oj,ieI). 

We  will  need  the  following  conditions: 

(AO)  For  each  n  e  N  and  each  i  e  J ,  limsup^^  N/’n(t)/t  <  oo  a.s. 
(A1 )  The  spectral  radius  of  matrix  P  is  strictly  less  than  1 . 

(A2)  For  all  i  e  A, 


A ,‘Hnx)  +  /x”(nx) 

sup  sup  - — - — - 

neN  xe /{£  n\l  T  W) 


<  OO. 


(A3)  For  all  i  e  A,  there  exist  continuous  functions  a,  (x)  and  /i,  (x)  such 
that 


xUnx)  .  ,  ti-(nx) 

^  A  X)  , 

n  n 

uniformly  on  compact  subsets  of  R+,  as  n 
x  e  R/, 


-x  Ri(x) 

oo.  Furthermore,  for  all 


A.(x)  —  P/x(x)  =  0, 

where  A.(x)  =  (A.j(x)  ,  i  e  K)  and  /x(x)  =  (/l(x)  ,  i  e  A)  . 

(A4) 


sup  sup 

neJV  x6j?y 


1 

Vn(l  +  |x|) 


|A"(Vwx)  —  R[zn(^/nx)  |  <  oo. 


Downloaded  by  [Chihoon  Lee]  at  1 1:56  06  February  2015 


Non-Markovian  State-Dependent  Networks 


49 


(A5)  There  exists  a  Lipschitz-continuous  function  a  ( x)  such  that 


—=(Xn(^/nx)  —  RiLn(yfnx))  — >  a(x) 
fin 


as  n  — >  oo  uniformly  on  compact  subsets  of  III 
(A6)  As  n  — »■  oo, 


K 
+  • 


((f\0),NA'n,  N°’n,  <&”)  =>  (A),  WA,  WD,  W*) 

where  A)  is  a  random  A-vector,  WA  =  ( WA ,  i  e  K) ,  WD  =  ( W/’ ,  i  e 
K) ,  and  =  ( Wfi  ,  i  e  K)  are  continuous-path  stochastic  processes 
with  trajectories  in  respective  spaces  B([0,  oo),  MK),  B([0,  oo),  RK), 
and  B([0,  oo) ,  1RKxK). 

Condition  (AO)  is  needed  to  ensure  the  existence  of  a  unique  strong  solu¬ 
tion  to  the  system  of  equations  (2.1a)-(2.1d);  see  Lemma  2.1.  It  is  certainly 
fulfilled  if  Na'  "  is  a  renewal  process  and  is  almost  a  consequence  of  condition 
(A6)  in  that  the  latter  implies  that  NA'n(nl) / (nl)  =  1  in  probability. 

Part  (Al)  is  essentially  an  assumption  that  the  network  is  open  and  under¬ 
lies  the  existence  of  a  regular  Skorohod  map  associated  with  the  network 
data  asserted  in  Proposition  2.1.  The  linear  growth  conditions  (A2)  and 
(A4)  are  needed  to  ensure  the  tightness  of  certain  processes.  Conditions 
(A2)  and  (A3)  appear  in  Mandelbaum  and  Pals 1 11 1 .  The  requirement  that 
X(x)  —  fl/i  (x)  in  (A3)  together  with  condition  (A5)  defines  a  critically  loaded 
heavy  traffic  regime.  Condition  (A6)  is  an  assumption  on  the  primitives.  The 
components  of  WA  corresponding  to  i  f  J  vanish.  Conditions  (A2)-(A5) 
are  fulfilled  if  the  following  expansions  hold: 

Xn(x)  =  nX^fix/n)  +  finX^  (x/fin)  and  pZ(x) 

=  n/i^fix/ri)  +  yfin/jL^  (x/fin),  (2.2) 

where  X  1 1  and  fiy>  are  continuous  functions  satisfying  the  linear-growth 
condition  such  that  A(1)(x)  =  A/z(1)(x),  and  A(2)  and  /x(2)  are  Lipschitz- 
continuous  bounded  functions.  If  the  functions  in  (2.2)  are  constant,  then 
one  obtains  the  standard  critical  loading  condition  that  (Xn  —  R/in)  /fin. 

A. 2  —  |U2  as  n  — >  oo;  cf.  Reiman t19^. 

Lemma  2.1.  Let  condition  (AO)  hold  and  max!eXsup;(,€Kx(A"(x)  + 
ti”(x) )/(l  +  |x|)  <  oo.  Then,  given  Qf( 0),  Nf’n,  N-y ”,  and  <I>"  equa¬ 
tions  (2.1a)-(2.1d)  admit  a  unique  strong  solution  Q_n,  which  is  a  Z+ -valued 
stochastic  process. 
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The  proof  is  provided  in  the  appendix.  In  order  to  state  the  main  result, 
we  have  to  recall  some  properties  of  the  Skorohod  map. 


Definition  2.1.  Letxp  £  B([0,  oo) ,  JRK)  be  given  with  xp  (0)  £  M+.  Thenthepair 
(</>,  q)  £  ID)([0,  oo),  HlK)  x  B([0,  oo),  HlK)  solves  the  Skorohod  problem for  xp  with 
respect  to  JR+  and  R  if  the  following  hold: 

(i)  4>(t)  =  xp(t)  +  Rift)  £  JR+,  for  all  t  >  0; 

(ii)  fori  £  K,  (a)  qf  0)  =  0,  (b)  qi  is  non-decreasing,  and  (c)  ly  can  increase  only 
when  p  is  on  the  ith  face  of  JR that  is,  /0°°  1  {fa^^ojdrifs)  =  0. 

Let  Dka([0,  oo),  JRk)  =  {xp  £  D([0,  oo),  MK)  :  xp{0)  £  JR !+}.  If  the  Skoro¬ 
hod  problem  has  a  unique  solution  on  a  domain  D  C  B^y  ( [0,  oo) .  IRK) , 
we  define  the  Skorohod  map  T  on  l)  by  T  (t/r)  =  f  .  The  following  result 
(see  Harrison  and  Reiman ^  and  also  Dupuis  and  Ishib4!)  yields  Lipschitz 
continuity  of  the  Skorohod  map. 

Proposition  2.1 .  Under  assumption  (A1 ),  the  Skorohod  map  T  is  well  defined  on 
Dka  ( [0,  oo) ,  UlK)  and  is  Lipschitz  continuous  in  the  following  sense:  There  exists  a 
constant  L  >  0  such  that  far  allT  >0  and  i/q ,  xp^  £  O  rk  ( [0,  00) ,  JRK), 

sup  |T(i/q)(0  -  T(t/r2)(0l  <  L  sup  \xpft)  -  xp<fit)\. 

is[0,  T]  «6[0,T] 

Consequently,  both  f  and  q  are  continuous  functions  of  xp . 

The  Lipschitz  continuity  of  the  Skorohod  map  and  of  the  function  a(x) 
imply  that  the  equation 


X(t)=r(Xo+  f  a(X(s))ds  +  M(-)](t), 

Jo 


(2.3) 


where 


K  K 

Mft)  =  Wf  (k,(0)0  +  K(iu(")t)  ~  - pji)WP^j(0)t), 

7=1  7=1 

(2-4) 

has  a  unique  strong  solution  (cfb4]).  For  t  >  0  and  i  £  K,  let  X’1  (I)  = 
Qf\t)/yfn..  We  also  define  X=  (X(t),  t  >  0)  and  Xn  =  ((X”(£),  i  = 
1 ,  2,  . . . ,  K) ,  t  >  0) .  The  proof  of  the  following  theorem  is  given  in  the  next 
section. 
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Theorem  2.1.  Let  conditions  (A0)-(A6)  hold.  Then  Xn  =>■  X,  as  n  — >  oo. 

Most  of  the  results  on  diffusion  approximation  in  critical  loading  (see, 
e.g.,  Harrison  and  Reiman ' 7  and  Kushner' 1 1 ')  formulate  the  heavy  traffic 
condition  in  terms  of  rates  that  are  on  order  one  and  then  consider  scaled 
processes  Q,l(nt) /  y/n.  In  the  scaling  here,  as  in  Yamada^'1^  and  Mandelbaum 
and  Pats ' 11 ' ,  the  time  parameter  is  left  unchanged,  and  the  factor  of  n  is  ab¬ 
sorbed  in  the  arrival  and  service  rates.  This  is  more  convenient  notationally; 
however,  in  the  application  to  generalizedjackson  networks  in  Section  4,  we 
work  with  the  conventional  scaling.  It  may  be  instructive  to  note,  though, 
that  if  one  looked  for  limits  for  processes  Qn(nt)  / y/n,  then  the  analogues  of 
expansions  (2.2)  would  be 

Xn(x)  —  A(1)(x/n)  +  (1  /  y/n)  X^  (x/y/n)  and 

fin(x)  =  /z(1)(x/w)  +  (1  /y/n) (x/y/n), 

whereas  the  assumptions  of  Yarnada 121  would  amount  to  the  expansions 

Xn(x)  =  X^  (x/y/n)  +  (1  / yfn)X^  (x/yfn)  and 
fin(x)  =  /x(1)  (x/y/n)  +  (l/y/n)fi^  (x/y/n). 

Theorems  1  and  2  in  Yarnada'21  obtain  diffusion  processes  with  state- 
dependent  drift  and  diffusion  coefficients  as  the  limits.  Theorem  1  concerns 
the  Markovian  model.  It  is  required  that  there  be  at  least  one  nonzero  exter¬ 
nal  arrival  process.  The  arrival  and  service  rates  at  a  station  may  depend  on 
the  queue  length  at  that  station  only.  Theorem  2  concerns  ajackson  network 
with  external  arrival  processes  being  Poisson  processes  with  state-dependent 
rates.  In  the  proof  of  Theorem  2,  the  process  d/(t)  claimed  to  be  a  locally 
square  integrable  martingale  on  p.  980  does  not  seem  to  have  the  asserted 
property.  The  model  itself  is  defined  by  postulating  certain  martingale  prop¬ 
erties  of  the  arrival,  service,  and  customer  transfer  processes.  No  description 
in  terms  of  the  primitive  processes  is  provided,  nor  is  the  issue  of  the  assump¬ 
tions  being  self-consistent  addressed.  The  gap  is  filled  in  by  Mandelbaum 
and  Pats' 11 although  the  authors  admit  the  proof  is  missing  technical  de¬ 
tail;  see  p.  623  in  Mandelbaum  and  Pats' 11 '.  Mandelbaum  and  Pats ' 11  give 
few  details  with  regard  to  the  existence  and  uniqueness  of  a  solution  to  (1.1) 
under  linear  growth  conditions  on  the  rates.  Mandelbaum  and  Pals ' 11  and 
Kushner' 1 1 '  allow  the  routing  matrix  to  be  state-dependent.  Mandelbaum 
and  Pats' M  appeal  to  Theorem  5.1  and  Corollary  5.2  in  Dupuis  and  Isliii''’  to 
substantiate  the  existence  and  uniqueness  for  the  associated  Skorohod  prob¬ 
lem;  however,  those  results  assume  bounded  domains,  so  they  do  not  apply. 
The  authors’  attempt  on  p.  628  to  recast  the  problem  as  a  time-dependent 
reflection  is  unconvincing.  Kushner^11",  in  their  proofs  of  Theorem  1.1  on 
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p.  309  and  Theorem  2.1  on  p.  318,  relies  on  their  Theorem  5.1  on  p.  123  and 
Theorem  5.2  on  p.  124,  which,  in  turn,  are  based  on  Theorem  2.2  in  Dupuis 
and  I  sh  i  i 11 1 .  However,  those  results  pertain  to  reflection  directions  that  are 
constant  on  the  faces,  so  they  do  not  apply  to  state-dependent  reflection 
directions.  Kuslmer' 1 1  does  not  address  the  issue  of  the  model  being  well 
defined  either.  Nor  are  we  convinced  by  the  substantiation  of  the  martingale 
properties  claimed  to  hold  on  p.  310.  Besides,  the  hypotheses  of  Theorem 
1.1  on  p.  309  and  Theorem  2.1  on  p.  318  of  Kushnert11^  are  missing  the  con¬ 
dition  of  the  drift  and  diffusion  coefficients  being  Lipschitz  continuous.  The 
functional  central  limit  theorem  on  p.  584  in  Mandelbaum  and  Pats ' 11 ' ,  for 
the  case  of  critical  loading,  is  missing  conditions  (A4)  and  (A5).  The  proof 
given  in  that  paper  is  carried  out  for  a  special  case  where  those  conditions 
are  met.  On  the  other  hand,  the  condition  that  the  first  moments  of  the 
initial  queue  lengths  be  finite  assumed  by  Mandelbaum  and  Pats 111  can  be 
left  out. 


3.  PROOF  OF  THEOREM  2.1 

We  assume  conditions  (A0)-(A6)  throughout  this  section.  We  introduce 
the  “centred”  processes  as  follows:  For  i  e  K  and  t  >  0, 

+Mf’n{t)  -  M?’n{t), 

where 

M?'n{t)  =N*'n(j\”(Qn(s))ds^  -  j\^(Qn(s))ds, 

K 

7=1 

and 

=  Af n  (J‘  tf(Qn(s))lms)>0}ds)  -  jT'  tf(QJ\s))lms)>0]ds 

-  J‘  Hnj(Q,n(s))lms)>0}ds\ 


(3.1a) 

(3.1b) 

(3.1c) 


(3. Id) 
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We  can  rewrite  the  evolution  (2.1a)  as 


QI\t)  =  QP(  0)  +  f 
Jo 

+  M”(t)  +  [. RYn(t )] 


K 


K(Qn(s))  +  I>fi/*"(Qn(*))  -/*?(£"(*)) 

7=1 


ds 


where  Yn(t)  =  ( Y”(t )  ,  i  e  K)  and 

Y?(t)  =  f1  lms)=0} unorw)  ds,  i  e  (3.2) 

Jo 

We  note  that  t  >  0)  is  a  continuous-path  non-decreasing  process  with 

17(0)  =  0,  which  increases  only  when  Cl"  (J)  =  0,  i.e.,  /0°°  Ipyui^oi-dL/^/)  = 
0  a.s.  Let 


an(x)  =Xn(x)  -  Rnn(x).  (3.3) 

Then  the  state  evolution  can  be  expressed  succinctly  by  the  following  vector 
equation: 

Q_n\t)  =  Qn(0)  +  f  an(Qn(s))ds  +  Mn(t)  +RYn(t),  t  >  0,  (3.4) 

Jo 

where  Mn{t)  =  (M”(t)  ,  i  e  J£).  It  can  also  be  described  in  terms  of  the 
Skorohod  map: 

<T(t)  =  T  (dn( 0)  +  J  an(Qn(s))ds  +  Mn(-))  ( t ),  f  >  0.  (3.5) 

The  following  tightness  result  is  essential. 

Lemma  3.1.  The  sequence  of  processes  (Mn  (t)  /  yfn,  L  >  0)  isC-tight. 

Proof.  Let 


K 

en(x)  =  l  +  ^o<(x)  +  *"(*)) .  A”(*) 

8=1 


dTW 

6>n(x)  ’ 


k”(x) 


and 


*?(*) 
0"(x) ’ 
(3.6) 


rn(t)  =  inf 


(«))  dii  >  t 


(3.7) 
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We  note  that  r n{t)  is  finite-valued,  strictly  increasing,  absolutely  continuous 
with  respect  to  Lebesgue  measure,  drn(t)/dt  =  1  /9n(Q_n(rn(t))  for  almost 
all  t,  and  r "(/)  — >  oo  as  t  — >  oo;  cf.  p.  307  in  Ethier  and  Kurtz161.  Substitution 
into  (2.1a)-(2.1d)  shows  that  the  process  Qn  =  (Qn(t)  ,t>  0)  defined  by 
Qn(t)  =  Qn{ xn{t))  satisfies  a.s.  the  equations 

QJ'(t)  =  &"(0 K(Qn(s))ds^ 

+  £  *nn  (<n  (jf  A”(^(^))i{p»w>0}^)) 

-N?'n(f‘w2n(s))l{(L »0)>o}^)  ,  t>  0.  (3.8) 

Also,  <2”( 0)  =  (7"(0).  Let  f  ”(t)  =  inf{s  :  xn(s)  >  t}  represent  the  inverse  of 
r”.  We  have  that 


an(t)  =  Qn(tn(t)). 


(3.9) 


One  can  also  see  that 


f  n(t)  =  inf  •{  5 


=  /‘ 


1 


0"(Q”(«)) 


du  >  O  . 


i=l 


t=l 


A 


i=  1 


By  (3.8),  on  noting  that  A."(*)  <  1, 

A'  A 

£&"«)  <£4”(o)  +  ££w(o. 

By  (3.11)  and  (A6), 

lim  lim  sup  P 

By  (A2),  (3.6),  (3.10),  and  (3.11),  for  some  H  >  0, 

P(t "(f)  >  nT) 


(sup£  -  Q™{ns)  >  —  0. 


n 


(3.10) 


(3.11) 


(3.12) 


H(n+  Ef=!  QT( 0)  +  Ef=i  N?’"(nu)) 


du  <  t  I  . 


(3.13) 
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Since  Q/*(0) /  n  — 0  and  Y/Li  N^’n(nu) / n  — >  Ku  in  probability  as  n  - 

oo,  the  right-hand  side  of  (3.13)  goes  to  zero  for  T  >  eKHt.  Since  by  (3.9), 


P  [sup  V''  -  Q]l(s)  >  r  J  <P(in(t)>neKHl)  +  P  [  sup  V"'  -  Q™{ns)  >  r  J  , 

V  &  i=i  n  f  \s<e™‘  i=1  n  ) 


we  conclude  by  (3.12)  that 


lim  lim  sup  P  (  sup  —  (dj'is)  >  r  }  =  0. 

r— >oo  .  ..  \  „  ,  ‘  J  n  I 


n— >oo  y  s<l  M 


(3.14) 


It  follows  by  (A2)  that,  for  i  e  K, 


lim  lim  sup  P 


(l(-WQ?(s)) +  -»?((£(*)))  ds 
Jo  V  n  n  ) 


>  r  =  0  (3.15) 


and  that,  for  S  >  0,  e  >  0,  T  >0, 


el+S  / 1  j 

lim  lim  sup  P  (  sup  J  ^-A-f(Qf(5))  +  -£t”(Qj”(s))  }  ds  >  e  J  =  0. 


a-»o 


ft— >00  \<6[0 


n 


(3.16) 


By  (3.1b),  we  have  that,  for  y  >  0,  8  >  0,  e  >  0,  T  >0,  and  r  >  0, 


/ 

sup 

s,«e[0,T]: 
\  h-t|<« 


-J=. Mf’n(t )  -  ^=Mf'n(s) 
n  Jn 


>  Y 


<P  (f  ^X”(Q”(s))  ds  >  r  )  +  P 


/  f,+s  1 

sup  / 

\te[0,T]Jt  n 


-*"( dn{s))ds>e 
n 


+P 


/ 


sup 

s,«s[0,r]: 

y  |s-t|<e 


n ,  .  —A,  n 

N]  W-N;  (s) 


>  Y 
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By  (A6),  the  process  WA  being  continuous,  (3.15),  and  (3.16), 


lim  lim  sup  P 

i->o 


/ 

sup 

s,  <€[(),  T]: 

V 


Mf'n{s) 


>  Y 


=  0. 


Hence,  the  sequences  of  processes  (M.f’n(t)/y/n,  t  >  0)  are  C-tight.  A  similar 
argument  shows  that  the  sequences  of  processes  (Mf'n(t)/^/n,  t  >  0)  and 
(Mf’n (t) / <Jn,  t  >  0)  are  C-tight,  so,  by  (3.1a),  the  sequence  of  processes 
( Mn\t)/yfn. ,  t  >  0)  is  C-tight.  □ 


We  identify  now  the  limit  points  in  distribution  of  m"  —  ( A/ " (/) / ^fn, 
t>  0). 

Lemma  3.2.  The  sequence  of  processes  Mn  converges  in  distribution,  as  n  — »■  oo, 
to  M. 


Proof.  From  Lemma  3.1,  Mn(t)/n—>  0  in  probability  uniformly  over 
bounded  intervals.  By  (A2)  and  (3.3),  for  some  H  >  0,  for  all  n 
and  x,  \an(nx)\  <  Hn{\  +  \x\) .  By  (3.14),  the  sequence  of  processes 
(/J (l/ri)an(Qn(s))ds,  I  >  0)  is  C-tight.  By  (3.5),  (A6),  Prohorov’s  theo¬ 
rem,  and  the  continuity  of  the  Skorohod  map,  the  sequence  of  processes 
(Qf  (l) / n,  t  >  0)  is  C-tight.  If  (q(t) ,  t  >  0)  is  a  limit  point  in  distribution, 
then,  by  continuity, 


q(t)  =  r 


5))  —  Rti(q(s)))  els 


( t )• 


Since  by  (A3),  X(x)  —  R/j.(x)  —  0,  we  must  have  that  q(t)  —  0,  which  implies 
that  the  sequence  Qf  (t)  / n  tends  to  zero  as  n  — >  00  in  probability  uniformly 
on  bounded  intervals.  By  (A3)  and  (3.3) ,  f(‘  (1/ n)  an(  Qn(s) )  els  — >  0  in  prob¬ 
ability.  Since  by  (3.4)  and  R  being  nonsingular,  Yn  is  a  continuous  function 
of  (Qn( 0)  +  fj  an(Qn(s))ds  +  Mn\t),  t  >  0),  we  have  that  Yn{t)/n — >  0  in 
probability  uniformly  over  bounded  intervals,  so  by  (3.2),  for  i  e  K, 

t 

lx"(Qn(s))l{Q’<(s)=o}ds  0  in  probability  as  w— >-00. 


(3.17) 
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We  also  have  by  (A3)  and  the  convergence  of  Q_n(t) / n  to  the  zero  function 
that 

t 

^(Q,n(s))  ds  —*■  7,(0) t  in  probability  as  n >,—*■  oo  (3.18a) 

and 


/  P”(Qn(s))  ds  -*  pbi(.0)t  in  probability  as  n o o.  (3. 

Jo 


1  rt 
n 


18b) 


Since  by  (A6),  N®'n(nt) / n  — >  /  in  probability  uniformly  over  bounded 


intervals  as  n  — ►  oo,  by  (2. Id),  (3.17),  and  (3.18b), 

mt) 


n 


Hi  (0)  t  in  probability  as  n  — >  oo. 


(3.19) 


On  recalling  the  definitions  in  (3.1b)-(3.1d),  we  conclude  from 
the  convergences  in  (A6),  (3.18a),  (3.18b),  and  (3.19)  that  the  pro¬ 
cesses  (MA,n / y/n,  MB'n / yfn,  MD-n / yfn)  jointly  converge  in  distribution  to 
( Ma,Mb,Md ),  where  Mf  (t)  =  Ilf  (A. ,-(0) 0 ,  M?(t)  =  £f=1  W£(/i,(0)f), 
Mf(t)  =  ^f~i (Sij  —  p ji)W?(Hj(Q)t).  Therefore,  by  (3.1a)  and  (2.4),  the 
processes  m"  converge  in  distribution  to  M.  □ 

Proof  of  Theorem  2. 1.  We  note  that  by  (3.5), 


xn(t)  =  r  xn(0)  +  /  — 


1  ^-an{sfnXn 
Jo  s/n 


( s))cls  +  M  (•)  ( t ),  t  >  0.  (3.20) 


By  the  Lipschitz  continuity  of  the  Skorohod  map,  (3.3) ,  and  (A4) ,  for  T  >  0 
and  suitable  H  >  0  and  L  >  0, 

f1  1 

sup  \Xn(t)\  <  L|X”(0)|  +L  sup  /  —\an(JnXn{s))\ds 
<e[0,T]  te[0,T]Jo  Vw 

+L  — sup  \Mn(t)  | 

V»  «€[0.r] 

<  L|X"(0)|  +LH  f  (l  +  sup|X”(s)|)  dt 
Jo  V  s<t  ) 


+L  — sup  \Mn{t)\. 

Vn  isfo.r] 
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Gronwall’s  inequality,  the  convergence  of  the  Xn(0)  in  (A6),  and  Lemma 
3.2  yield 


lim 

r— >CO 


lim  sup  P 

n— >oo 


(sup 

<S[0,T] 


\Xn{t)\>r 


=  0, 


so,  by  (3.3)  and  (A5),  the  sequence  of  processes 
(/q  an(y/nXn(s)) /y/n.  ds  ,  t  >  0)  is  C-tight. 

By  (3.20),  the  convergence  of  the  X"(0),  Lemma  3.2,  (A5),  Prohorov’s 
theorem,  and  the  continuity  of  the  Skorohod  map,  the  sequence  of  processes 
(Xn(t),  t  >  0)  is  C-tight  and  every  limit  point  ( X(t ),  t  >  0)  for  convergence 
in  distribution  satisfies  the  equation 


L 


X(t)  =  r[X(0)+  a(X (s)  )ds  +  M (•)  )  (t) ,  t  >  0. 


The  uniqueness  of  a  solution  to  the  Skorohod  problem  implies  that  X(t)  = 
X(t).  □ 


4.  GENERALIZED  JACKSON  NETWORKS  WITH 
STATE-DEPENDENT  RATES 

In  this  section,  we  consider  an  application  to  generalized  Jackson 
networks  in  conventional  scaling.  We  assume  as  given  mutually  indepen¬ 
dent  sequences  of  i.i.d.  nonnegative  random  variables  { u'Ari),  i  >  1}  and 
{ v'h  ( n) ,  i  >  1}  for  j  e  J  c  K  and  k  G  IK.  For  the  nth  network,  the  random 
variable  ul  ( n)  represents  the  ith  exogenous  interarrival  time  at  station  J, 
while  v'k  ( n)  represents  the  ith  service  time  at  station  k.  The  variables  pij  rep¬ 
resent  the  probabilities  of  a  customer  leaving  station  i  being  routed  direcdy 
to  station  j,  which  are  held  constant.  The  routing  decisions,  interarrival  and 
service  times,  and  the  initial  queue  length  vector  are  mutually  independent. 

We  define 

k'!  =  (E [«;.(n)])_1  >  0,  a]  =  Var («).(»))  >  0,  j  e  J,  and 

Hnh  =  (E[^(n)])_1  >  0,  si  =  Var (t/J(n))  >0,  k  e  K, 

with  all  these  quantities  assumed  finite  and  the  set  J  assumed  nonempty.  It 
is  convenient  to  let  A."  =  1  and  an-  =  0  for  j  $  J . 

Let  N^'n(t)  =  max-JY  :  ^J.=1  ulj(n)  <  t}  for  j  e  J  and  N^’n(t)  = 
max{i'  :  wKn)  —  0  f°r  ^  e  A2  We  may  interpret  the  process 

(Ay1' ”( t) ,  t  >  0)  as  a  nominal  arrival  process  and  the  random  variables  v'k  ( n) 
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as  the  amounts  of  work  needed  to  serve  the  customers.  Suppose  that  ar¬ 
rivals  are  speeded  up  (or  delayed)  by  a  function  X”(x),  where  i  £  J,  and 
the  service  is  performed  at  rate  fi'l (x) ,  where  k  £  K,  when  the  queue  length 

vector  is  x.  As  in  Section  3,  we  let  Nf’n(t)  =  [/J  and  A"(x)  =  0  for  i  $  J .  In 
analogy  with  (2.1a)-(2.1d)  the  queue  lengths  at  the  stations  at  time  t,  which 
we  denote  by  Q]1  ( t) ,  are  assumed  to  satisfy  the  equations 

&"«)  =  4"(0)  +  A" (0  +  W  -  A"(0 - 

K 

mt) 

7=1 

D?(t)  =NP’n(^J‘  K(Qn(s))lms)>0}ds^  , 

where 

m 

*%(*)  =  ExjiW’ 

i=i 

with  {(/ "■(/),  i  —  1,  2, . . . ,  K),  l  =  1,2,...}  being  Bernoulli  random  vari¬ 
ables  that  are  mutually  independent  for  different  j  and  l  and  are  such  that 

PUJid)  =  1)  =  Pji- 

If  we  introduce  the  random  variables  QJ'(d)  =  QJl{nt),  A'-(t)  = 
A'-  (nt),  B?(t)  =  B”(nt),  D?(t)  =D?(nt),  Nf’n(t)  =  N*’n(t/X”) ,  N°’n{t)  = 

N^’n (t / /x”) ,  and  3>"-(m)  =  and  functions  A.”(x)  =  nl™X”(x)  and 

/x”(x)  =  n/z"/x"(x),  then  we  can  see  that  they  satisfy  equations  (2.1a)-(2.1d). 
Condition  (AO)  holds  as  Nf'n(t) / 1  —>■  1  and  N®’n(t) /t  ->  1  a.s.  as  t  — >  oo. 
Assume  that  Q"(0)  / yfn.  =>•  Xq,  and  for  k  e  K,  j  e  J , 

X”  -*  \j,  a]  -»•  aj, 

/ 1  k  ^  /i  /.'  ■  ^  ^  • 

as  n  — »■  oo,  and  also  that 

max  supE(n^(n))"+e  +  max  supE(V.(n))2+'  <  oo  for  some  e  >  0. 

keK  n>  1  7S>7  n>l 
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Then  condition  (A6)  holds  with  Wj4  =  j Bj  for  j  £  J,  Wf(t)  =  0  for 
j  J,  =  ^/skl-ikB'k  for  k  £  K,  and  with  W®  being  a  A'-di  mens  ion  al  Brow¬ 
nian  motion  with  covariance  matrix  E  W®(t)W®(t)  =  (pijSjk  ~  pijpik)  A 
where  B‘j  and  fijP  are  standard  Brownian  motions  and  the  processes  Bj, 
BijP  and  are  mutually  independent. 

Let  us  assume  that  the  following  versions  of  conditions  (A2)-(A5)  hold: 

(A2)  For  each  i  £  K, 


sup  sup 


X"(nx)  frj(nx) 


nelN  xeMK  \1  +  M  1  +  1*1 


<  oo. 


(A3)  There  exist  continuous  functions  A;(x)  and  ft  Ax)  such  that 
A ”(nx)  —¥■  Xi(x),  jlf(nx)  —>■  jlj(x) 


uniformly  on  compact  subsets  of  1R+,  as  n  — >  oo.  Furthermore,  for  x  £ 


lK 

v+> 


X(x)  —  R/jl(x)  =  0, 

where  A.,-(x)  =  AA,(x)  and  /Z;(x)  =  /tiA*(x), 

(A4) 

| Xn (yfn.x)  —  RfXn(s/nx)\ 
sup  sup  V  n - - — — — ; -  <  oo  . 


»€iV  x£]rk 


l  +  |x| 


(A5)  There  exists  a  Lipschitz-continuous  function  a ( x)  such  that 
y/n{Xn  {yfnx)  —  KjZn(*/nx))  — a{x) 


as  n  —¥■  oo  uniformly  on  compact  subsets  of  1R+,  where  A™(x)  =  A”A”(x) 
and  jl"(x)  =  /r”/l”(x). 

Then  the  process  M  in  (2.3)  and  (2.4)  is  a  A-dimensional  Brownian 
motion  with  covariance  maUix  A  that  has  enuies  for  i  £  K, 


An  =  ij(0)X^ai  +  A* (0) yu-f ^ (1  -  2 pa) 

K 

+  ^  '  A  j  (0)  B  jP  /i(l  P  ji  p  ji/l  jSj), 

j= 1 
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and  for  \  <  i  <  j  <  K, 


A..  — 

**  l]  - 


K 

fi  /  (0  )  1 1  .  Sf  p  ij  flj  (0  )  Jlj  S  jp  jj  +  'y  fl  /;  (  0  )  j  l  f/p  /.  ip  /.  j  (  1  jl  ^  5  k  ) 

k=l 


An  application  of  Theorem  2.1  yields  die  following  result. 

Corollary  4.1.  If,  in  addition  to  the  assumed  hypotheses,  condition  (A1 )  holds,  then 
the  processes  ( Qn(nt )  /  yfn,  t  >  0)  converge  in  distribution  to  the  process  (X(t) ,  t  > 
0)  that  satisfies  the  equation 

X{t)  =  T  (%>  +  fo  a(X(s))ds  (t), 

where  B(-)  is  a  K- dimensional  standard  Brownian  motion. 

Remark  4.1.  The  conditions  on  the  asymptotics  of  the  arrival  and  service 
rates  essentially  boil  down  to  the  assumptions  that  the  following  expan¬ 
sions  hold:  k"(x)  =  A(1'(x/n)  +  A(_>  (x/y/n)/y/n  and  JZn(x)  =  ~jl(l){x/n)  + 
/Z(2) {x/y/n)/*Jn with  suitable  functions  A(1),  X"\  /Z(1),  and  p(2). 

Remark  4.2.  If,  in  addition,  the  assumption  of  unit  rates  is  made,  that 
is,  A ,j(x)  =  1  for  j  e  J  and  jX'^x)  =  1  for  k  G  K,  then  the  limit  process  is 
a  ^-dimensional  reflected  Brownian  motion  on  the  positive  orthant  with 
infinitesimal  drift  a( 0)  and  covariance  matrix  A,  and  the  reflection  matrix 
R  —  /  —  PT,  as  in  Theorem  1  of  Reiman^19]. 

Remark  4.3.  In  order  to  extend  applicability,  one  may  consider  inde¬ 
pendent  sequences  of  weakly  dependent  random  variables  { ul- ( n) ,  i  >  1}, 
{ v'k(n),  i  >  1}  for  j  e  J  c  K  and  k  e  K.  Under  suitable  moment  and 
mixing  conditions  which  imply  the  invariance  principle  (cf.  Herrndorf 
Peligrad^17^,  Jacod  and  Shiryaev^),  Corollary  4.1  continues  to  hold. 

5.  A  CASE  STUDY:  A  TAIMDEM  QUEUE  OF  HUMAN  SERVERS 

In  production  systems  with  human  servers  productivity  may  depend  on 
the  workload.  With  low  workload,  the  productivity  may  be  low  because  of 
“laziness,”  whereas  when  the  workload  is  too  high,  the  productivity  may  be 
adversely  affected  by  the  associated  “stress”;  see  van  Ooijen  and  Bertrand1 15 
and  references  therein.  (More  generally,  the  Yerkes-Dodson  law  has  it  that 
the  relation  between  performance  and  arousal  is  given  by  an  inverted  U- 
shaped  curve.)  Motivated  by  empirical  evidence,  van  Ooijen  and  Bertrand[15] 
use  a  triangle-shaped  function  to  express  the  dependence  of  productivity  on 
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2 


FIGURE  1  The  tandem  queue. 

the  workload  and  investigate  the  effect  of  controlling  the  arrival  rate.  We 
will  adopt  a  similar  model.  We  consider  two  service  stations  in  series  (see 
Figure  1 ) . 

Customers  who  depart  from  station  1  are  routed  to  station  2  and  leave  the 
tandem  queue  upon  completion  of  service  at  station  2.  Exogenous  arrivals 
occur  at  both  stations.  In  the  nth  tandem  queue  the  service  rate  at  station 
i  when  the  queue  length  at  that  station  equals  xt  is  represented  by  /r''(x,j , 
where  i  —  1,2,  and  the  compression  of  the  interarrival  times  between  the 
exogenous  arrivals  is  represented  by  A.”( x\ ,  xy).  The  iid  service  requirements 
of  individual  jobs  at  station  i  are  assumed  to  have  mean  1  /ft,  and  variance  y; 
hence,  the  service  completion  rate  at  station  i  is  ii,n'l(xl) .  The  iid  nominal 
interarrival  times  have  mean  1  /A.,-  and  variance  a,  so  that  the  actual  arrival 
rate  is  A.,A."(x i,  xg). 

We  assume  further  that 


The  functions  jin  and  /ly,  have  triangle-shaped  graphs  as  depicted  in 
Figure  2.  The  exogenous  arrival  rates  are  controlled  in  order  to  ensure  high 
utilization  of  the  servers,  on  the  one  hand,  and  not  to  increase  the  waiting 
time  excessively,  on  the  other  hand,  for  which  purpose  the  critical  loading 
condition  is  maintained: 


WAii(xi,  X2 ) 
A.2A12  (xi ,  X2) 


In  order  for  these  equations  to  have  a  nonnegative  solution  for  ^(xi,  xy), 
we  need  to  require  that  min*,  /i  ly/iiyixy)  >  m ax X|  ii\\ji\\  ( X] ) . 

Let  Q”(t)  denote  the  queue  length  at  station  i  at  time  t.  As¬ 
suming  that  QJ’i 0)  =  0,  we  have,  by  Corollary  4.1,  that  the  processes 
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FIGURE  2  Dependence  of  the  service  rate  on  the  queue  length. 


(Ql’(nt)  / «Jn,  Q£(nt)/y/n)  converge  in  distribution  to  the  -valued  process 


/Xi(£)\  [ '  ( X2(s))  /  1  0Wi;i2i(Xi(s))\\ 

\X2(t)J  J0  {{x2X22(X1(s),X2(s))J  V-1  1  J\^22(X2(s))JJaS 


where  B\  (l)  and  B2(t)  are  independent  standard  Brownian  motions,  Y\  (t) 
and  Y2(t)  are  non  decreasing  continuous  path  processes  such  that  Y,  (0)  =  0 
and  fl  Xj(s)  dYi(s)  =  0,  for  i  —  1,2,  and  the  2x2  matrix  A  has  entries 


An  — A.nA.^ai  + A22  —  X\2X^a2 -\-  id\2iD'2s2 -\-  i1\\I-l\si, 
A2i  =  A\2  =  —  /Xn/X^i. 


APPENDIX 

Proof  of  Lemma  2. 1.  The  proof  is  an  adaptation  of  the  one  in  Puhalskii  and 
Simon ' IK  (Lemma  2.1)  and  employs  the  approach  of  Ethier  and  Kurt/'1’ 
(Theorem  4.1,  p.  327).  We  saw  in  the  proof  of  Lemma  3.1  that  if  Qn{t) 
satisfies  (2.1a)-(2.1d),  then  Qn(t)  =  Qn{xn{t)),  where  r n(t)  is  defined  by 
(3.6)  and  (3.7),  satisfies  (3.8).  We  now  show  that  if  Q_n{t)  satisfies  (3.8),  then 
Qn(xn(t)),  where  xn(t)  is  defined  by  (3.10),  satisfies  (2.1a)-(2.1d).  The 
random  variable  xn(t)  is  well  defined  for  all  t  a.s.  because,  by  hypotheses,  for 
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a  suitable  constant  Ln,  9n(x)  <  Ln(  1  +  |x|)  so  that,  by  (3.11), 
1  1  T  1 


[ 


-du> 

0n(Qn(u))  L 


> 


-  f- 

rZ  Jo  1 

If 

Ln  Jo 


+  Ef=i  Q?(«) 


-da 


1 


LnJ°  i  +  ELct+EL^'w 


du.  (Al) 


Since  limsup^^  N^'n(t)/t  <  oo  a.s.,  the  rightmost  integral  in  (Al)  tends 
to  infinity  as  s  — *■  oo  a.s.,  hence,  so  does  the  leftmost  integral,  which  proves 
the  claim.  In  addition,  f  n(t)  is  absolutely  continuous,  stricdy  increasing, 
drn(t)/dt  =  6n(Qn(Tn(t))  a.e.,  and  rn(t)  ->  oo  as  t  — >  oo  a.s.  In  particular, 
Jo  0n(Qn(rn(s))  ds  <  oo.  Substitution  in  (3.8)  shows  that  the  random  vari¬ 
ables  QJ(in(t))  for  i  e  K  satisfy  the  equations 

Q]l(rn(t))  =  QP(0)  +Nf-n(j\?(Qn(Tn(s)))ds) 

+  E<f"«(<"(f1^(^"(s)))1{S;™)>  o}^)) 

i  «/  0 


1=1 

-AE 


?’n(fo  tf(0.n(tn(s)))l min{s))>0}ds),  t>  0. 


Therefore,  existence  and  uniqueness  for  (2.1a)-(2.1d)  holds  if  and  only  if 
existence  and  uniqueness  holds  for  (3.8).  The  existence  and  uniqueness  for 
(3.8)  follows  by  recursion  on  the  jump  times  of  Qn.  More  specifically,  we 


define  the  processes  Q^1  =  (Q_,u(t),  t  >  0)  with  Q_nX(t)  =  i  e  K) 


by  Q”’°(t)  =  Q?(0)  and,  for  i  =  1,  2, . . .,  by 

oj-\t)  =  (an(o) + <’”(1  ^(Q^-^i))^) 


+ 


1=1  0 


)) 


-NP’n(j‘  mor-t-Hs)): 


{4  h)>o}( 


j)+- 


(A2) 


The  processes  Q_n’1  are  well  defined,  are  -valued,  and  have  right  contin¬ 
uous  trajectories  with  left-hand  limits.  Let  t  ■  represent  the  time  epoch  of 
the  £ th  jump  of  with  r”’°  =  0.  By  (A2),  Qnl  (I)  =  Qn'°( 0)  if  t  <  r "■ 1 . 
Therefore,  Qn,2(t)  =  <2”’X(0  if  t  <  r”’1.  It  follows  that  (Q'!l(t),  t  >  0) 
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and  ( Qll’~(t ),  t>  0)  experience  the  first  jump  at  the  same  time  epoch 
and  the  jump  size  is  the  same  for  both  processes,  so  r”1  <  r”’2  and 
(T”-2(0  =  Qn  l(t  A  r”’1)  for  t  <  r"'2.  We  define  Qn(t)  =  Q"'°(0)  for  t  <  xnA 
and  Qn(t)  =  Q"’1  (t)  for  r 1  <  t  <  r"2.  Since  the  processes  A'/'1' "  have  unit 
jumps,  we  have  that  (3.8)  holds  fort  <  r”2.  By  induction,  for  arbitrary  l  e  JV, 
we  obtain  that  <  xnX+1  and  Q_nX+l{t)  =  Q_nX{t  A  xnX)  for  t  <  xnX+1.  We 
let  Qn(t)  =  QnX{t)  for  xnX  <  t  <  xnX+1 .  The  process  Q_n  is  defined  consis- 
tendy  and  (3.8)  holds  for  t  e  U^;1[r”^_1,  xnX) .  If  xnX+1  =  oo  for  some  i, 
then  we  let  Q_n\t)  =  QnX(t)  for  all  t  >  xnX. 

Suppose  that  xnX  <  oo  for  all  t.  Then  Q_n{t)  has  been  defined  for  all 
t  <  r”  °°  =  lirri/^oc  xn,t  and  satisfies  (3.8)  for  these  values  of  l.  We  now  show 
that  rn  0°  =  oo.  The  set  of  the  time  epochs  of  the  jumps  of  Qn  is  a  subset  of 
the  set  of  the  time  epochs  of  the  jumps  of  the  process  Q_n  =  ( Q_n\t ),  t  >  0), 
where 


an(t )  =  K(Qn(s))ds)  +  ^An(^Ar(aB(^)1{4-w>0}^) 


Since  the  process  Q_n  has  infinitely  manyjumps,  so  does  the  process  (J".  Since 
X”(x)  <  1  and  n'-(x)  <  1,  the  lengths  of  time  between  the  jumps  of  Qn  are 
not  less  than  the  lengths  of  time  between  the  corresponding  jumps  of  the 
process  (£f=i  +  £f=i  N?'\t) ,  t  >  0)  .  The  process  (£f=i  N?’n{t)  + 

,  t  >  0)  having  infinitely  manyjumps  and  being  finite  for  all  t, 
the  time  epochs  of  the  jumps  of  that  process  tend  to  infinity  as  the  jump 
numbers  tend  to  infinity.  Thus,  xn'°°  =  oo  a.s. 

The  provided  construction  shows  that  Q_n  is  a  suitably  measurable  func¬ 
tion  of  ()”(0),  NA' ",  ND'n,  and  O",  so  it  is  a  strong  solution.  The  unique¬ 
ness  is  proved  similarly.  More  specifically,  suppose  Q_n  is  another  solution 
to  (3.8).  Let  r  ■  represents  the  time  epoch  of  the  fth  jump  of  Q_n.  We 
also  let  f”’°  =  0.  Suppose  that  Qn(t)  =  Qn(t )  for  t  <  x"J:  which  is  true  for 
t  =  0.  Then  Qn(t )  =  Qn{t)  =  Qn{xnX)  for  t  <  xnX+1  A  xnX+[.  Hence,  the  in¬ 
tegrals  from  0  to  xnX+1  A  f"J+l  on  the  right  of  (3.8)  are  the  same  for  Qn 
and  Qn,  which  implies  that  Qn(xnX+1  a  f  nX+] )  =  Q_n(xnX+l  A  xnX+1).  Since 
one  of  the  processes  jumps  at  xnX+1  A  f",<  + 1  the  other  must  jump  too,  so 

^n,£- 1-1  |-1  |— j 
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